1 of 36 



/ 45 J 7 



150 



CCD 
MODULE 



100- 



102 
S 




CCD 
CONTROLLER 



PREVIEW 
ENGINE 



■104 



108 



COMPRESSION 



=0 



DSP 



■122 fn> 



128 



110 



SDRAM |C 
CONTROLLER 



IMAGE BUFFERS 



120 



124 



126 

L 



IMaging 






Extension 




VLC 


(iMX) 







7T 

I 130 



160 

J_ 



SDRAM 



NTSC/PAL 
ENCODER 

T 

106 



ARM 



MEMORY 

r 



132 



DSP SUBSYSTEM 



FIG. 1a 



=0 



=>TV 



USB 
CORE 



I/O 
BLOCK 



-142 
-140 



CCD 
MODULE' 



PREVIEW ENGINE 



CONTROLLER 



rrn _ BURST MODE 
^ u COMPRESSION 



=0 



DSP SUBSYSTEM 



DSP 



I 



L 



SDRAM ! 
CONTROLLER j_. 



i 



IMAGE BUFFERS 



ARM 



MEMORY 



IMaging 






Extension 




VLC 


(iMX) 







16. 



SDRAM 



NTSC/PAL 
ENCODER 



=£>TV 



USB 
CORE 



I/O 
BLOCK 



2 of J>0 



o 











LU 






O 


ENGI 

RAMI 




82 

^ CO 


RESS 


LlJ 1— 




l_ LU 

coQ^ 


OMP 






ct: ^ 


0 




CD O 
O 


1 

LU 


Q. 




O 




o 

lo- 



co 

CD 










0 


<c 








ZD 




oc 

LU 


CPl 


0 


X 

LU 





o 



CM 


USB 
Xciever 


5 


0 


1 




1— 
0 


TH RAM) 






USB 
(WITH 
RAM) 


ZD 


0 




TIMEF 


S: 


IrDA (Wl 


SERI A 



m 



en 



CD \— 
ZS1 

o 
o 



1 — 


az 


Q_ 


LU 

1 


ZD 


1 


Q£T 


0 


ct: 


on 


LU 




I — 


2: 




0 




0 



32K BYTES 
RAM 






•4 — ► 


JTAG 



o 



CM 



O 



0 


U- 

co 


0 


CO 



95 



to 



O 



CD 
O 

CO 
O 







LU 


i 




Q_ 


^ ° 


O 


g o< 






zo 




1 — 


CO 





Ql oh 

CO CM 



I 





i 






CCD 


MODULE 



CNI 



s 

Q 

o 



CM 



00 

™ CO 



CO o 

LU 



O 

o 



CO 

LU^ 
CD ^- 



0 


INE | 


1 

> 


0 

LU 



1 cvi- 



— CO 
O Z 
<£ LU 




~3 



>- 








O Lu- 




CO 


LU 


CO 




0 




SH 


CF 



V 

00 



CC CC 
LU O 

co lZ 




CCD 
MODULE 




PREVIEW ENGINE 









CCD 
CONTROLLER 




BURST MODE 
COMPRESSION 



=0 



DSP SUBSYSTEM 



DSP 



SDRAM 
CONTROLLER 



l i- J 

* IMAGE * 





FIG. 3 a 



47.8 cyc/pxl= 
12230 cyc/16x16 pxl 




B- 



iMX USES BUFFER A 



A MB — B 

iMX USES BUFFER B l 
VLC, SDRAMC, 



VLC, SDRAMC, 
DSP USE BUFFER B DSP USE BUFFER A 

FIG. 3 b 



PREVIEW 
ENGINE 



CCD 
MODULE 



4o 



3E 



CCD 
CONTROLLER 



BURST 
MODE 
ENGINE 



DSP SUBSYSTEM 



^z:::::::::::^ SDRAM 



SDRAM 
CONTROLLER 








IMaging 






Extension 




VLC 


(iMX) 









ARM 




MEMORY 




f 437 



NTSC/PAL 
ENCODER. 

in 



DMA 



USB 
CORE 



H$>TV 



CFC 



CCD 
MODULE 



PREVIEW ENGINE 



CCD 
CONTROLLER 



BURST MODE 
COMPRESSION 



DSP SUBSYSTEM 




SDRAM 
CONTROLLER 



SDRAM 



IMaging 






Extension 




VLC 


(iMX) 







1 NTSC/PAL 
ENCODER 



CCD 
MODULE 



5 of .36 



4*7 



PREVIEW ENGINE 



CCD £j 
CONTROLLER 



BURST MODE 
COMPRESSION 



DSP SUBSYSTEM 



DSP 



t. ■ 



SDRAM ! 
CONTROLLER I. 



t IMAGE BUFFERS t 



O 



SDRAM 



— -s^TV 

NTSC/PAL 

ENCODER 



ARM 



MEMORY 



IMaging 
Extension (iMX) 



VLC 



IE 



0> 



USB 
CORE 



I/O 
BLOCK 



FIG. 6 



R 


G 


R 


G 


G 


B 


G 


B 


R 


G 


R 


G 


G 


B 


G 


B 



FIG. 7a 



Ye 


Cy 


Ye 


Cy 


G 


Mg 


G 


Mg 


Ye 


Cy 


Ye 


Cy 


G 


Mg 


G 


Mg 



FIG. 7 b 



lf( IN[17]=1 or IN[16]=1) then 
OUT[9..0]=0x3FF; 

els6 

0UT[9..0]=IN[15..6]; 
end if; 



dota[9..0]- 



PVGAIN[7..0]- 



lf( IN[17]=1 or IN[16]=1 or IN[15]=1) then 
OUT[7..0]=0xFF; 

else 

OUT[7..0]=IN[H..7]; 
end if; 



v 18 BITS r 


CLIP 


10 BITS^ 




S 





0^$ 



gainl[7..0]- 
goin2[7..0]- 
gain3[7..0]- 
goin4[7..0]- 

goin_selector_sw 



18 BITS. 



0x00010 -J 
white_bolonce_gain[7..0] 

FIG. 8 



CUP 



dato_out 
^[7..0] 



6 of 36 



doto[7..0]— 
goin[7..0] * 

FIG. 9 a 



doto[l5..0] 



CLIP 



dota_out[7..0] 



K 



lf( data[l5]=1 or data[H]=1 ) then 

data_out=OxFF; 
else 

doto_out=data[l3..6]; 
end if; 



OUTPUT 



0FFSET3 
0FFSET2 

OFFSET 1 



FIG. 9 b 




*■ INPUT 



data_in[7..0]- 

areol[7..0]- 
orea2[7..0]- 
areo3[7..0]- 

gainl[7..0] 
. goin2[7..0] 
gain3[7..0] 
goin4[7..0] 

0x00 
offset2[7..0] 
offset3[7..0] 
offset4[7..0] 



16 BITS r 




AREA 
DETECTOR 





dota[12..0] 



CLIP 




dato_out 
[7..0] 



A , » r7 nl S 255 (data[12..0]3255) 
data_out[7..0]=| dQt(j[7 0] OTHERS) 



FIG. 9 c 



CO 
CO 

oo 



en 



O CD 

ii 



comp2RGB 



CO 

55 
oo 



>- 



a i 










o 




o 


O 


GO 





COLOR 
ADJUSTMENT 



CD <Z 



OO 





k i 




CP 


o 


o 


O 


CD 





COLOR 
SELECTION 



to 

CD «— 
I 

OO 



nj 



oo 

o. 

o 



CNJ 



o 

on 



Z <; <j 

< LdQ. 

Of U. t= 
> — 



CO 

j— ■ 

CD 

to 



on 



ri 



CO 

o 



CO 

'o 



o 



3 
O 



"O 



CNJ 
"O 



o 
o 



HORIZONTAL 
DOWN SAMPLING 



CO 
I — , 

CD 
CO 



on 



o 



a n 



O 



Q 
O 



O 

o 



o^ 

CL 



Q 
CO 



HORIZONTAL 
INTERPOLATION 



CO 
) — 

CD 
OO 



.o 

O 



HORIZONTAL 
NOISE FILTER 



CNI 
.O 

o 




CO 
I — 

CD 
OO 



O 

^3 
"o 
-o 



• 

RAW DATA 



G 00 



G 02 



Boi 



B03 



G 04 



Gil 



G 13 



G 20 



G 22 



B 21 



HORIZONTAL Jl 
INTERPOLATION v 



G 00 



b 00 



Rio 



910 



G 20 



b 20 



901 



Boi 



10 



-11 



921 



B 21 



G 02 



b 02 



! 12 



912 



G 22 



b 22 



B 23 



903 



B03 



13 



G 13 



923 



B23 



G 24 



G 04 



b 04 



Rj 4 



914 



G 24 



b 24 



FIG. 10c 



NORMAL MODE 

-R 10 +2G 11 +2R 12 +2G 13 -R 14 
912= 4 

SIMPLE MODE 

G n +G 13 



912 = 



2 



data 
[7..0]' 



r-1 



r-1 



0x00 



"1 



r-1 



8 BITS 
8 BITS 



'/-HJNTP 
8 BITS 



8 BITS 



10 BITS 



'-1 - 



8 BITS 



x 2 + 1 


11 BITS 


x 1/4 





lf( IN[9]=1J then 
OUT[7..0]=0x00; 
else 

lf( 1N[8]=1) then 
OUT[7..0]=0xFF; 
else 

OUT[7..0]=lN[7..0]; 
end if; 
end if; 



H_INTP 



,10 BITS 


CLIP 




y 



data_out 
' [7..0] 



9 BITS 



r 12~BlTS 



FIG. 10d 



y ot 



OUTPUT OF HORIZONTAL INTERPOLATION 



901 



B 01 



G 02 



b 02 



r 11 



<12 



912 



921 



B 21 



922 



B 22 



VERTICAL ' fl 
INTERPOLATION V 




r 01 



901 
Boi 



r 02 



G 02 
b 02 



r 11 
Gil 



11 



R 12 
912 



12 



r 21 



921 
B 21 



r 22 



922 
B 22 



COLOR ADJUSTMENT J~l 
(NORMAL MODE) V 



— b Q2 -G 02 +b 22 -G 22 
b 12 = 5 912 



FIG. 10e 



r 01 
901 
BQ1 

ill 

921 
B 21 



r 02 
G 02 

b 02 

=1 

R 12 

912 
^12 

r ~22 

922 
B 22 



903 



B 03 



13 



G 13 



923 



B 23 



r 03 



903 
Bq3 



r 13 
G 13 



13 



r 23 



923 
823 



r 03 
903 
BQ3 

~n7 

^3_ 
b 13 

923 
B 23 



SIMPLE MODE 



r 01 
901 
BQ1 

~n7 

G 11 



B 



11 



r 21 

921 
B 21 



K 02 
G 02 
b 02 

R 12 

912 
b 12 

922 
B 22 



r 03 
903 
BQ3 

G 13 
ill 

IK 

923 
B23 



10 of. 36 



ii4 



OUTPUT OF HORIZONTAL INTERPOLATION 



Ye 00 


c yoo 






G 10 


m 900 


Y *20 


c y2o 





y e oi 


c yoi 






911 




y e 2i 


°y2i 





Ye 0 2 


c yo2 






G 12 


m 9l2 


Ye 22 


cy 22 





VERTICAL f] 
INTERPOLATION V 



Ye 00 


°yoo 


900 


m 900 


y e io 


cyio 


G 10 


m 9l0 


*20 


c /20 


920 


mg 2 Q 



y e oi 


C Y01 


Ye 02 


°yo2 


901 


mg 0 i 


902 


mg 02 


y e n 


cy n J 


y e 12 


c yi2. 


911 


Mg n 


G 12 


mg 12 


y e 2i 


C Y21 


Ye 22 cy 22 


921 


mg 21 


922 


mg 22 



COLOR ADJUSTMENT J~l 
(NORMAL MODE) V 



y e oo 


c yoo 


900 


m 900 


y e io 


cyio 


910 


m 9lO 


y e 2o 


c y2o 


920 


mg 20 



y e oi 


c yoi 


901 


m 901 


y e n 


cyn 


9TT 


m 9l1 


WT\. 


C Y21 


92T 


mg 21 



y e 02 


c yo2 


902 


mg 02 


y e 12 


c yi2 


912 


mg 12 


ye 22 


cy 22 


922 


mg 22 



Y e 1 1 = 



cy n = 



yepi+ye21 
2 

Cyoi +c y2i 



SIMPLE MODE 



y e oo 


c yoo 


900 


m 900 


y e io 


°yio 


G 10 


m 9l0 


Ye 2 0 


c y2o 


920 


mg 20 



y e oi 


cyoi 


901 


m 901 


y e n 


c yn 


911 


Mg n 


y e 2i 


CV21 


921 


mg 21 



o=g 11 +Mg 11 -ye 11 -cy 11 
ye?T=y e 11 + f 

cyTT=cyn+7 
9n=girj 

mg 11 =Mg 11 - j 



Y *02 


c /02 


902 


mg 02 


y e 12 


c yi2 


G 12 m 9l2 


Ye 22 cy 22 


922 


mg 22 



FIG. 10f 



HORIZONTAL INTERPOLATION OUTPUTS 



G 00 



b 00 



901 



BQ1 



G 02 



b 02 



! 10 



910 



11 



G 11 



G 20 



b 20 



921 



B 21 



NOISE FILTER = OFF 
L12T=R 12 

R_121=g 12 
L_101 = ^^ 



903 



B 03 



G 04 



b 04 



R 12 




r 13 




R 14 








912 




G 13 




914 






















VERTICAL' 


I 
















INTERPOLATION 


















G 22 




923 




G 24 








b 22 




B 23 




b 24 









L121 
R_ 1 21 
L101 
R_101 



NOISE FILTER = ON 

G 02 +2g 12 +G 22 
L_121=R 12 -g 12 + t 



R_121= 



L_101 = 



R_101= 



G 02 +2 9l2 +G 22 
4 

G 02 +G 22 



b 02 +b 22 



FIG. 10g 



VERTICAL 
INTERPOLATION 


1 1 01 

JU21 ^ 


COLOR 
SELECTION 


R_121 


L_101 


R_101 






R/Ye 
G/Mg 
B/Cy 
■g/G 



NOISE FILTER = OFF 
R/Ye=R 12 

G/Mg=g 12 

G 02 +G 22 



g/Cy= 
B/Cy= 



2 

b 02 +b 22 



NOISE FILTER = ON 

d/v d , Cq2 +2 912 +G 22 
R/Ye=R 12 -gi2+ 4 

G 02 +2 912 +G 22 



G/Mg= 
g/Cy= 
B/Cy= 



4 

G Q2 +G 22 
2 

b 02 +b 22 



2 



FIG. 1 0i 



left. 101- 
right. 101 

left. 121 
right. 121 




cfa.pattern- 



R/Ye 



COLOR EXTRACTOR 



cfa.pottern- 



cfa_pottern ■ 





g/G 



FIG. 1 Oj cfa.pottern 



COLOR EXTRACTOR 



14 of 36 



ZD 
O 
I 

CD 
>- 



0£ 



CD 



o 

I 

CP 



I 

o 

CX> 



O 



t 





y[7..0] — 

r 


I 


g[7..0] — 




G[7..0] — 





o3 o 

o 



c 

CD 

x: 

~ p 
o S 



o 

II 




/~ II co^ ^ 

. O CD O 

O CD — CD 

— jy? 

— CD 



O 
O 



CNJ 
X 



CO 
I — 

CD 

cr> 



O 



CNI 



OJ 


CNJ 


X 


X 






O 


'Mg[7. 


/Ye[7. 


o 





CNJ 
X 



00 
l — 

CD 



On 



CNI 



o 



CD 



CD 



CNI 



CNJ 


CNJ 


X 


X 



I 

X 



p p 
^ o 



•52 
o 



CN) 



On 



o 
I 



O 

II 

o 



CNI 



I— 

CD 
CNI 



it 

f 



R[7..0]- 



G[7..0]- 



B[7..0] 




coefl[7..0] 



coef2[7..0] 



coef3[7..0] 



coef4[7..0] 



coef5[7..0] 



coef6[7..0] 



coef7[7..0] 



coef8[7..0] 



F 
iJ 



x 1/64 



x 1/64 



0 (Yin<0) 
Y[7..0]=<j 255 (Yin>255) 
Yin[7..0] (others) 



Yin[l1..0] 



CLIP 



■Y[7..0] 



x 1/64 



x 1/64 



i (Cbin^-128) 

Cb[7..0]=<{ 255 (Cbin*127) 

Cbin[7..0]+128 (others) 





Cbin[!2..0] 



CUP 



offset_Cb[7..0] 



■Cb[7..0] 



x 1/64 



coef9[7..0] 



nl-J 



x 1/64 



i (Crin^-128) 
Cr[7..0]=-{ 255 (Crin*127) 
Crin[7..0]+128 (others) 

^: 



Crin[l2..0] 



CLIP 



offset_Cr[7..0] 



■Cr[7..0] 



FIG. 11a 



OUTPUT 
ENHANCEMENT SIGNAL 



t2~ 



0 



FIG. 1 1 b 




SLOPE SI 



INPUT 
EDGE. SIGNAL 



17 of 36 



FIG. 12 a 



BURST MODE 
COMPRESS AND 
DECOMPRESS 



DSP 



PREVIEW 
ENGINE 



CCD DATA 
BUFFER 



SDRAM 



EXTERNAL 
FLASH/CFC 





sc 


(RAM CONTROLLER 





< 










1 




IMAGE 
BUFFER 




DATA 
BUFFER 


— ► 


ARM 





OSD 



VIDEO 
ENCODER 



PERIPHERAL 
MODULES 

req [0] 
reqen [I] 
address [0] 
odata [0] 
idata [I] 
bs [0] 
rw [0] 
dten [0] 
elk [I] 



SDRAM 
CONTROLLER 

[1] req ras [0] 

[0] reqen cas [0] 

[I] address we [0] 

[I] odata cs [0] 

[0] idato bank [0] 

[I] bs dqm [0] 
[I] rw address [0] 

[I] dten data [10] 

[I] elk eke [0] 



TO 
SDRAM 



FIG. 12b 



IN 

_L 

DPCM 



HUFFMAN 
ENCODING 



ENCODED 
DATA 



HUFFMAN 
TABLE 



HUFFMAN 
DECODING 



I -DPCM 



"^1 

FIG. 13b 



MODULE- 



PREVIEW ENGINE 



TV ENC 



CCDC 



COMP ENGINE 
DECOMP ENGINE 



SDRAM 
CTRL 



IMAGE BUFFER +\0$> 

JTJ 

DSP iMX VLC 



J 

ARM 



C±C> SDRAM 



FIG. 13a 












CNJ 


> 






NO 






o 



GO 
CD 
OO 



i i 


i li 






0 


0 



'5 



—1 


UJ 


Q- 


0 


Tscy 


:nco 




l-LJ 





19 of 36 



XO.YO 



XO.YO 



O 



CO 
CO 

oo 



1 BIT BitMop 



X1.Y1 



XO.YO 



YCbCr 



X1.Y1 X1.Y1 
VIDEO DISPLAY 



FIG. 16 



XO.YO 






WIND0W1 






X1.Y1 


VIDEO DISPLAY 



FIG. 17 



122 



FIG. 18a 



DSP 




DSP EXT 




MEM l/F 



BUFFER A DATA 
512x64b 



BUFFER B DATA 
512x64b 



SDRAM 
CONTROLLER 

_L_ 



SWITCH NETWORK 



iMX 



iMX COEFF 
1280x48b 



•124 



VLC 



iMX GMD 
512x16b 



VLC QMAT 
512x16b 



•126 



VLC HUFF 
1024x32b 



OOOOh 

0080h 

7F80h 
8000h 
COOOh 



FF80h 
FFFFh 



PROGRAM SPACE 
(MP/MC_=0) 



RESERVED 
(OVLY=1) 



ON-CHIP 
32kword DARAM 
(OVLY=1) 



VECTORS 



EXTERNAL 



RESERVED 



RESET VECTOR 



OOOOh 
0060h 
0080h 

7F80h 
8000h 
COOOh 



FFFFh 



DATA SPACE 



MMR 



SCRATCH -PAD RAM 



ON-CHIP 
32kword DARAM 
(0VLY=1) 



RESERVED 



EXTERNAL 



RESERVED 



FIG. 18b 



20 of -36 




FIG. 19 



D_inp 

_L 



md_odd 



» 



md_odd 



~2J 

CUP 



CMP 



soLhigh 



7 



lookup.base 



] 



lookup_addr 

FIG. 21 



SHARED COEFF 
MEMORY MEMORY 

_J i_ 



ARBITER 



DATA 
MEMORY 
WRITE 



COEFF 
MEMORY 
READ 



DATA 
MEMORY 
READ 



DWR 



ADDRESS GEN 



DRD 



DRD 



INPUT FORMATTER 



DATA PATH 



COMMAND 
MEMORY 



DRD 



COMMAND 
DECODE 



EXECUTION 
CONTROL 



FIG. 20 



DSP EXT 
MEM l/F 



SHARED DATA 
MEMORY A/B 




FIG. 22 



32K WORD RAM 



DSP 



HP1B 



EM l/F 



BUS CONTROLLER 



ARM7TDM1 



EXTERNAL MEMORY 



FIG. 23a 



D[15:Q] 
cs_hpib 
cs_dspc 



HCNTLO 



O 



CD 
Q_ 



HCNTL1 



HD[7:0] 



5409 SUBCH1P 



HPI BLOCK 



HP1 BLOCK 
REGISTER 
SELECT 



HPIA[15:0] 




ADDRESS r 


32K 
WORD 
DARAM 




DATA 


HPIC[15:0] 


HP1D[15:0] 







FIG. 23b 



m • 





ARM PROGRAM/ 

r\ATA pnA/^r 

DATA SPACE 


OOOOh 


DSP DATA SPACE 


0000:0000h 


RESET VECTOR 




0000:0003h 


(BRANCH CODE) 
4 BYTES 


0060h 


MMR 


0080h 


SCRATCH -PAD RAM 


0000:0004h 


INTERNAL RAM 
- 32 KBYTES 


0000:7FFFh 


/ 




RlbtKVtD 


/ 


ON-CHIP 


0003:0000h 

AAA7 A(~rr~U 

0003:0FFFh 


INTERNAL 
PERIPHERALS 
4 KBYTES 


/ 


32K WORD DARAM 
(0VLY=1) 


0010:0100h 


RtbtKVtU 


/ 7F80h 




DSP MEMORY 


8000h 


RESERVED 


0010:FFFFh 


64 KBYTES 




IMAGE BUFFER 




DrcrD\/rr\ 
KtbtKVtlJ 










0200:0000h 
027F:FFFFh 


EXTERNAL FLASH 
8 MBYTES 








DrccD\/m 
KLbLKVtU 




RESERVED 


0400:0000h 
047F:FFFFh 


CFC 
8 MBYTES 


FFFFh 
















0800:0000h 
OFFRFFFFh 


SDRAM 
128 MBYTES 




FIG. 23c 



FIG. 24 



TDI 



TDO 
ARM7 TDM I 



TDI 



TDO 



C54X 



CO-EMULATION LOGIC 



TDI 

• nTRST 
•TCK 
■TMS 
-TDO 



EMU 1 
EMU 0 



CO 01 JO 



O CO o 

. — 1 



7S 



o 



o 
o 



U! 



Hi 

E 



ni 



Z 



CO 

CO ^ 
LU 2 

!< 



xnw 



in 



co 


<c 

Q 


LlJ 






—j 








t— o 




o 


II 


1— 

cr 
<t 


NTR 




CO 


o 


o o 
o 


o 



3<f 



1 CO 



CM 

o 

CO 

o 










o o 




g CO 










Lu£ 












Li_ CO 





or 




LU 
1 


o 
t— 


CO ^ 




;2 ^ 




CD t — 








o 







CO 
D£ 



O 

o 



Q 
<f 

CO 

or 

I 

o 



o 















ZD 








5: 



J 






CD 


o 


«t 




CD 


o 








TONE 




SCALING 


CD 










COLOR 


CORRECTION 






i i 




CD 


o 


en 






WHITE 




BALANCE 






i 




CD 


o 


or 






CFA 


INTERPOLATION 



£1 



L"t Ul JO 



STEP 1 



1 Q ! 



ii 



vie 



> D 



i is i 



} p r 



-Ci — 



B 



"~ •";] m 




ALREADY HAS BLUE 



5 ! DOES NOT HAVE BLUE 



B 



BEING INTERPOLATED 



FIG. 28 



1 1 

1 

^ « 

O 1 

1 




1 1 

1 1 

I ^ 1 

1 O 1 

1 1 

L---- J 


II 


W 
'////, 


i 1 

i i 
i c " 

1 O 1 

1 1 

L--_- J 


'////, 

'My 
Y//A 


( 1 

1 1 
1 ^ 1 
I O I 

1 1 

I- — - J 


i 1 

i i 
i ^ i 

1 O 1 

i i 


///// 

<w 


G 


w,. 


i 


1 1 

1 1 
I 1 

1 O 1 

1 1 


'////, 

'Ml 
'//£ 


i 1 

i i 
i ^ i 

1 o I 
i i 


i 1 

t i 

1 0 ! 

1 O 1 

i i 


1 


i 1 

i i 

1 1 
1 O 1 
1 1 
L-.--J 


i 



W7, 



ALREADY HAS GREEN 



— -, > 
• 



B 



I 

' J 

I 

1 

L-- 



W J JJ 



! G ! DOES NOT HAVE GREEN 



IS BEING INTERPOLATED 



5x5 FILTER 



B-2 

CLIP 
TT 



INTERPOLATED 
VALUE 



G1 



G2 



G3 



G4 



o 

1 

£ 

o 
o 



I I 

B-1 



FIG. 29 



DISCARD THE BIGGEST 
AND THE SMALLEST 



CM 



fas. 
L-J: 



U 




cnT 



c 
c: 



4- 
c 

CM^ 
+ 

cz 

+ 
c 

o 



CM ^ 

£^ + 
CM C 

is 

C C 

ii jT 

H- ■+■ 
c c 



or 

LU 
31 



—i o 



1 lO 

go 

^ II 



o" 

CM 



cr 




LU 


lO 


i — 


— 1 o 


Li_ 




1 


lO 


oc 


O 


Ll. 


II 


tn 


Ll. 
i . i 


TAP 


eoi 







CD 

+ 
c 

CD 

o 




if 



PO 

+ 

c 

CNI 

+ 
c 



a?* 



CM CM 
+ + 

c c 

«— " CM* 

L I I 

~ -J, £ 

+ o 

+ + 

a: c c 
«— " cm" 

i ii 

□e' "cT "cT 

^< — " CNI 
C . I I 

J. X J, 

^ o' 0^ 



CM CM 

cm" 

L I I 

+ CD O 

^ + + 

^< *— " cm" 

i ii 

£ CD CJ> 

*^ *^ 
^ «— ~ CM* 
C . I I 

Jl J. J. 
CD O 







o 








Lo 




o 




o 


II 




i 


LU 

m 




t — 




i — 


o 








OR 




3Z 










<c 


II 




I 


CO 


< 




O 






X 


en 






CD 





LU 

a: 



I — 
ZD 

o 



< 

I— 

ZD 



CM 

I 



) — 
O 



CD 



ZD 

GL 



o 



> 








> 




on 






on 


o 






o 








~£ 

LU 


LU 

~s 








kit 
Nt 






INE 


i 






i 


i 


i 


. i 


i 



CD 



CD 



ZD 
O 



O 



CM 

c 



^E 



CM 
+ 

^E 

CD 



^E 







1/2 









.G(m-1,x),.. 
.R(m-2,x)... 
..G(m,x),..-— NO USE 

.B(m-1,x),.. 

..G(m-2,x),.. — NO USE 



G(m-1,x) 



B(m-1,x) 



FIG. 33a 



.G(m+1,x),..— -NO USE 
..R(m,x),.. 



R(m,x) 



.G(m-1,x),..— NO USE 



■■B(m+1,x),..— -ffi H V2 | — »- B(m,x)=(B(m- 1 ,x)+B(m+ 1 ,x))/2 

..G(m,x),.. «-G(m,x) 

..B(m-1,x),.. FIG> 33b 



RAW DATA 
(CFA IMAGE) 



3 
o 

Q_ 



o 
o 



FIG. 34 



LINE MEMORY 



LINE MEMORY 



LINE MEMORY 



LINE MEMORY 



LINE MEMORY 



LINE MEMORY 



LINE MEMORY 



LINE MEMORY 







LU 
O 




ZD 
Q 




UJ 
OH 




LU 
CO 




ION 




AND 


— ^RED 


ION 


— ► GREEN 


3LAT 


— ►BLUE 


:rpi 








1 




<c 
o 








LU 
> 





01 jo 



..G(m,x).. 
.G(m-1,x).. 
.G(m-2,x).. 

.B(m-1,x).. 

..R(m,x).. 
,R(m-2,x)..' 





DIGITAL 
FILTER 






+ r 


k Delto_G(m-1 


^JT 


»~ 




1/2 




f 









G"(m-1,x) 



B'*(m-1,x) 



FIG. 37a 



.G(m+1,x)..- 
..G(m,x)..- 
..G(m-1,x)..- 

..R(m,x)..- 
,.B(m-1,x). 
..B(m+1,x). 





DIGITAL 
FILTER 








\ Delta_G(m,x) 




► 






1/2 


— — 4 




— <3 


i 







G"(m,x) 



R"(m.x) 



FIG. 37 b 




FIG. 38 



0.8 - 



OUTPUT o 6 
VOLTAGE 



0.4 - 



0.2 - 



20 




30 



40 50 60 
INTENSITY 



- FIG. 39a 



80 




. FIG. 39b 



90 100 



350 
300 
250 

OUTPUT 200 
INTENSITY 15Q 

100 
50 

•o, 




FIG. 40 



100 



200 300 400 500 
INPUT INTENSITY 



0 1 2 3 4 5 

1 I I I I I 



61 62 63 64 65 

I I I I I 



0 
i 



10 



61 



62 



63 

I 



FIG. 41a 



L 



A 



-1 0 1 

FIG. 41b 



10 



XXXXXXXXXXiXX 



xxxxxxxxxxxx 





FIG. 42a 



FIG. 42b 



8 



xxxxxxxx 



xxxxxxxx 



xxxxxxxx 



xxxxxxxx 



xxxxxxxx 



xxxxxxxxl 



FIG. 42 e 



B 





HISTOGRAM 






EQUALIZATION 








LUMINANCE - 




CONVERTER 



-432 
430 

•-LUMINANCE 



FIG. 43 



LUMINANCE- 





HISTOGRAM 
EQUALIZATION 








a EFFECT 


} 




— II — fr» 


CONVERTER 



•442 
444 

440 
LUMINANCE 



FIG. 44 




0 500 1000 1500 2000 2500 3000 3500 4000 

r 



FIG. 45 



TdO 



Td1 



i 



Td2 



Td3 



. I 



Tdm 



DECODING TIME 



FRAMEq 


FRAME 1 


FRAME 2 


FRAME 3 


* • • 


FRAME m 


PRESENTATION 

• ' ' 1 TIME 












i t 


. « 



AT 



2AT 



3AT 4AT 

FIG. 46a 



mAT 



TdO 



Td1 



Td3 i 



I Tdm 



I DECODING TIME 



FRAME o 


FRAME ! 


FRAME i 


FRAME3 


• • • 


FRAME m 


• • • 






> * < 






fc 



PRESENTATION 
TIME 



0 AT 2AT 3AT 4AT 

FIG. 46b 



mAT 



t 



I — * 
CO 
I — 

CO 



O 
LU 

O 
UJ 




BUFFER 0 (TP 0 ,S 0 ) 



• • • 



BUFFER n-1 (TPn.^Sn.-,) 



BUFFER n (TP n ,S n ) 



BUFFER n+1 (TP n+1 .S n+1 ) 



BUFFER N-1 (TPn.-i.Sm.-i) 



FIG. 47 





5 

CO 



fa- 
ns 



SLICE 0 
SLICE 1 
SLICE 2 



SLICE n 



MACROBLOCK 



FIG. 49 



SDRAM 
CONTROLLER 



=0 



B1STREAM 
BUFFER 



SDRAM 

FIG. 50 



IE 



VIDEO 
DECODER 




DISPLAY 



FIG. 51a 




Ps 



BUFFER STATUS 
AFTER DECODING A UNIT 



S3 



RE-FILLED BUFFER BEFORE 
y A DECODING THE NEXT UNIT 



Pd 




Ps=Pd 



Ps=Pd 



BUFFER STATUS 
AFTER DECODING A UNIT 



^ RE-FILLED BUFFER BEFORE 



DECODING THE NEXT UNIT 

FIG. 51b 



JJ 01 JU 



( START ) 



S n =TP n =0, O^ngN 
n=t=m'=Am=0 



S n =1, SKIP Am FRAMES 
DECODE FRAME m' 



Ts=t, n++ 
COMPUTE TP n , m', Am 




FIG. 48 



-j 



n=N fl t=0, T=0 




( STOP ) 



Ts=max|T, TP n+1 | 
COMPUTE TP n , m\ Am 




S n =0, SKIP Am FRAMES 
DECODE FRAME m' 




YES • 



S n =1, T=t, n+ + 



T=t - Td+=DT- m=m ' 



36 of 36 



♦ 



LEFT BUFFER 



RIGHT BUFFER 



M^^Mk^ BUFFER STATUS AFTER INITIALIZATION 



Ps 



STATUS 0 



I 





Ps 



Ps 




BUFFER STATUS ARER DECODING A UNIT 



BUFFER STATUS AFTER RE-FILLING BUFFER 



STATUS 1 



I 



NOT-FULL 




^^FULL^ 






S 






^noT-fuU^ 



BUFFER STATUS ARER DECODING A UNIT 



BUFFER STATUS ARER RE-FILLING BUFFER 



Ps 



STATUS 2 



I 



NOT-FULL 



Ps 



^NOT-FULL 

Y /; /,///// 




BUFFER STATUS ARER DECODING A UNIT 



%l7\\W^yA BUFFER STATUS AFTER RE-FILLING BUFFER 



Ps 



STATUS 3 



FIG. 52 



